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ABSTRACT 



A method of upscaling a color image in a first color space 
format to a second color space format may include the 
following. A color image in a first color space format may 
be transformed to a second color space format. At least one 
color plane of the transformed color image may be upscaled, 
the at least one color plane corresponding to one of the color 
space dimensions of the second color space format, to 
provide an upscaled color image in the second color space 
format. Of course, many other embodiments other than the 
preceding embodiment are also within the scope of the 
present invention. 

20 Claims, 4 Drawing Sheets 
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METHOD OF UPSCALING A COLOR IMAGE practiced without these specific details. In other instances, 

well-known methods, procediires, components, and circuits 

RELATED APPUCATION have not been described in detail so as not to obscure the 

present invention. 

This patent application is related to concurrently filed ^ previously indicated, for a color image, typically each 

U.S. patent application Sen No. 09/461,068, tiUed "Method pixel signal value is represented by three independent color 

of Converting a Sub-Sampled Color Image," by Acharya, components, such as a rod component, a green component, 

assigned to the assignee of the current invention and herein and a bhic component, for example, although the invention 

incorporated by reference, is not limited in scope to these color components or to a 

10 particular color space format. Frequently, it may then be 

BACKGROUND desirable to both upscale that color image and transform it 

This disclosure relates to color conversion and, more ^^^^^^ space format. Therefore, for a color image 

particularly, to converting from a color image in a first color captured by an imaging device, each pixel may be repre- 

space format to an upscaled color image in a second color rented by a 24 bit color signal, 8 bits for each of the red, 

space format. 15 ^^^^ ^^^^ components, for example, although the 

. . , , • . f * . * invention is not restricted in scope by this example. In order 

As IS well-known, m a vanety of circumstances, it is a , , i j i • . • * 

^ . 1 \ iT • ; 1 to produce such a upscaled color image contammg 24 bit 

desirable to apply a technique to convert from one color • t • c • • .t_ i 

^ J a I. VI " " ' - . „ , signal mformation compnsmg the three color components, 

Space format to another color space format for a full color ^ i • i • * i u u* i* 

" „ , *u • * u *u for example, signal interpolation, such as bi-hnear or 

image. For example, a camera or other imager, that has the i.- l- • . r r i i_ i j 
^ ' , . J *u • 70 bi-cubic mterpolation, for example, may be employed, 

capabihty of ere atmg a colorimage, may produce the image ^ r™_ . * n 

, ^ . ^ LI 7r.^T^\ 1 r * r Thesc appioachcs, however, are computationally quite 

m a red, green, blue (RGB) color space format. However, for i i- *• • 

" u J • 1.1 * 1 r complex, at least in part because the upscaling operation is 

transmission, it may be desu-able to comply with a fre- r j J n i i • *u i r 

' , , ^ - , „ , • f f • apphed to all three color planes m the color space format, 

quently employed International Telecommunications Union i * j * ui *u w • i J 

/tt^ts • . J J i_ .t_ Tr-.£io* J J and may also tend to blur the resultmg miage. If an upscaled 

(IIXT) CO cnmumcatio IIS standard, such as the H. 263 Standard , . • i . * i • .i • . c ^« • ^ 

y ^/.^ ^ ' r^^^^^^ ri.-t^u.^ oumuoiu ^^j^^ ^ obtamed m this manner, a transformation mto 

for Video Coding Suitable for Video Teleconferencmg. Such , .a: f i r * .i. ^ » j i. 

, . . . 1, 1 j-ir . 1 f . a different color space format may then be employed, such 

standards typically employ different color space formats, u j • ui * i -^u * - ' 

I- iAiT/ 1 f . ft. • as where it may be desirable to comply with certam image 

such as the YUV color space format or some other variation. -c ^- j/ . j j i r 

, . T J • i_i ^ 1 1 processing specifications and/or standards. Examples oi 

In addition, it is sometimes desirable to upscale the color f, ■ i j ^ j j -j j i_ ^t, tt-it /t * i 

. . J , j.*^ 1 . these include standards provided by the ITU (International 

image so that it may be viewed on larger displays or similar . ^. tt • \ ic^n n ^ ^- i j j 

J • TT f ^ .1 i_ ^ V Telecommunication Union), or ISO (International Standard 

other devices. Unfortimately, such upscahng may result m a „ • \ t -i • •/ • i . • i . * i 

J . J . cl 1^-1 ^ 1 Organization), Likewise, it is also typical to represent color 

blurred image and is often relatively computational . . . At . Vt • j * 

1 L- L J • 1.1 • images m a Luminance-Chrommance (L-C) form m order to 

complex, which imdesirably may consume excess power m j .u* • .i. * jj. 

, • 1- J . reduce the storage requirements by removme redundant 

situations where power is a limited resource, such as on a , . i • r .i. . l * • *u • 

, , , K. 1 • i_ n color signal mformation that may be present m the image, 

laptop computer, for example. These issues may be further . • n . • *u i^t. • i * ^ 

^ I- 1 r ^^'j typically present in the Chrommance color components. One 

exacerbated if, in addition, to upscalmg, color space format •* i * j ^ , , ^ -^i. a ^ 

, \ J - T * 1 1 A J widely used L-C color space format is the Y-Cr-Cb 4:2:0 

conversion, as previously described, is to take place. A need . ^- • * j • * 

. ^ , , • . ^ 1. • r representation. To accomphsh this, it is common to decimate 

exists for relatively efincient techmques for creatmg an *^ , . , K„ . ' . n i_ j 

, , i-x 11. 1 • the chrommance planes (Cr and Cb) usually by dropping 

upscaled full color unage of good quauty from a color image ^ « j i r*/ u * rr o 

^, ^ . every alternate row and column of the chrommance planes 

m a different color space format. i j i • a . h • 

40 of the upscaled color image. As a result, potentially signifi- 



BRIEF DESCRIPTION OF DRAWINGS signal information may be lost when this process 

is applied. 

The subject matter regarded as the invention is particu- Disadvantages of current techniques to produce an 

larly pointed out and distinctly claimed in the concluding. upscaled color image in a different color space formation 

portion of the specification. The invention, however, both as 45 include (1) the loss of signal information, sometimes pro- 

to organization and method of operation, together with ducing blurring, as a result of applying the particular 

objects, features, and advantages thereof, may best be under- technique, and/or (2) high computational complexity, which 

stood by reference to the following detailed description also may typically consume excessive power when 

when read with the accompanying drawings in which: employed or performed by a computing engine, and/or (3) 

FIG, 1 is a schematic diagram illustrating one tedinique 50 internal buffering large enough to store the signal informa- 

for producing an upscaled full color image converted from tion for the upscaled color planes and also after color space 

another color space format; conversion but before decimation. These disadvantages will 

FIG. 2 is a schematic diagram iUustrating a technique of explained in more detail hereinafter. An embodiment of 

upscaling a color image in accordance with the present upscaling a color image in accordance with the present 

invention* 55 invention may avoid at least some of these disadvantages. 

FIG. 3 'is a schematic diagram iUustrating an embodiment ^ ^ well-known, a typical interpolaUon method or 

of the discrete wavelet transform (DWT); « weU-known bi-lmear or bi-cubic 

„^ ...... ... . . mterpolation, for example, generates a upscaled color digital 

FIG 4isaschematicdiagramillustraUngoneaspectof an ^ comprising three color planes, red, green, and blue, 

embodmient of upscalmg a color miage in accordance with ^ ^^j, 2Mx2N from an MxN fuU color image. Of 

ttie present mvention. course, these are just examples of a color upscaling inter- 

DETAILED DESCRIFnON P°''''°? P"^"^: V"^"" ^ " """"i " °I ^f^'^S, 

mterpolation techniques that may be employed. In typical 

In the following detailed description, numerous specific approaches, the color signal information of the neighboring 

details are set forth in order to provide a thorough under- 6S pixels are employed to generate the desired color signal 

standing of the invention. However, it will be understood by information, although, of course, the invention is not limited 

those skilled in the art that the present invention may be in scope in this respect. FIG. 1 illustrates a process in which 
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a full color image is used to produce an upscaled color image 
in another color space format. 

Subsequently, for the approach illustrated in FIG. 1, an 
upscaled color image is converted to a Luminance - 
Chrominance (L-C) color space format, such as Y-Cr-Cb 
4:2:0, to meet color specifications or criteria, such as 
provided, for example, in the H.263 ITU standard, applied in 
video conferencing applications. Of course, the invention is 
not restricted in scope in this respect The color space format 
conversion from 24 bit RGB color space format to 12 bit 
Y-Cr-Cb 4:2:0 color space format is done using the follow- 
ing transformation: 

KC/,J)=0-299/?(/^4O.587G(y)+0.114J(AJ) 

Cr(/^=-0.16S(ff(7.J>-a.331G(/;j)4O.500fi(/^ [1] 
Cfc(//)=0.50Q«(/J)-0.419G(AJ)-O.0815(/^ 

where R,G, and B denote different color planes in the RGB 
color space format, Y, Cr, and Cb denote different color 
planes in the Y-Cr-Cb 4:2:0 color space format, and I,J 
denote pixel locations 

The Cr and Cb image planes are then, in this particular 
approach, decimated by dropping alternate rows and col- 
umns from each chrominance plane (Cb and Cr), as illus- 
trated by Cr' or Cb' in FIG. 1. This generates the Y-Cr-Cb 
image in 4:2:0 color space (12 bit color) format. The reason 
for 12 bits for each pixel in Y-Cr-Cb 4:2:0 color space is that 
for every 4 luminance (Y) pixels (1 byte each for each 
luminance value) in this format there is 1 value (1 byte each) 
for eadi corresponding chrominance channel (Cb and Cr in 
this example). Hence, 4 full color pixels in this format are 
represented by 6 bytes or 48 bits. As a result, each pixel in 
Y-Cr-Cb color space format is considered to have 12 bits. 

Assume that the average number of operations per pixel 
for upscaling in each color place is "K", when using a known 
upscaling technique. Then, for each pixel location, the total 
number of operations to upscale the full color image using 
this approach is 3Kx(MxN), where *'x** denotes multiplica- 
tion. The total number of multiplications for color conver- 
sion is 9(2Mx2N), or 36(MxN). Therefore, ignoring the 
computation for decimation, the typical number of opera- 
tions for upscahng followed by color conversion is more 
than 3(K+12)MxN. Likewise, the memory buffer size 
empbyed for hardware and/or software implementation of 
this approach is 3(2Mx2N) or 12MxN. 

FIG. 2 illustrates an embodiment in accordance with the 
present invention. Of course, the invention is not limited in 
scope to this particular embodiment. A full color image in a 
first color space format is first transformed to a second color 
space format, such as Y-Cr-Cb color ^ace format, without 
applying a decimation operation, as in FIG. 1. Then, at least 
one of the color space dimensions in the transformed image 
(luminance in Y-Cr-Cb formal) is "up-scaled" to provide a 
full color image in the second color space format. In this 
context, the term up-scaled refers to a process applied to a 
color plane of a color dimension in a particular color space 
format whereby the color signal information in the color 
plane is retained, but spread or distributed over a color plane 
of larger dimensions. For example, for the embodiment 
illustrated in FIG. 3, the Y so plane of size MxN is upscaled 
using a discrete wavelet transform (DWT) based upsampling 
technique in order to generate a MxN Y — Cb — Cr 4:2:0 
color image. Of course, the invention is not limited in scope 
to using the discrete wavelet transform and many other 
upsampling techniques may be employed, such as bi-Iinear 
interpolation, bi-cubic interpolation, weighted averaging, 
etc. An advantage of this particular approach is that the 
chrominance plane is not decimated and, hence, there is no 
significant loss of signal information, as would typically 
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occur. Even if another upsampling technique were 
employed, a additional loss of signal information would 
occur using the approach of FIG. 1, instead of this particular 
embodiment, for example, 

J Of course, this particular embodiment does not use 
bi-linear or bi-cubic interpolation to upscale. In the particu- 
lar embodiment, the properties of the inverse discrete wave- 
let transform (IDWT) are employed. See, for example, M. 
Antonini, M. Barlaud, R Mathieu, and I. Daubechics, 
"Image Coding Using Wavelet Transform", IEEE Transac- 

10 tions on Image Processing j Vol. 1, No, 2, April 1992, pp. 
205-220. FIG. 3 is a schematic diagram that illustrates the 
decomposition of an image (I) using the DWT into 4 
subbands, LL, LH, HL, and HH. The LL subband, size 
M/2xN/2 in this particular embodiment, may be considered 
at a subsampled or decimated version of the original image 
(I) of size MxN. In this embodiment, the other subband, HL, 
LH and HH, comprise high frequency subbands and, 
therefore, include high frequency edge signal information 
and noise. A close approximation of the original image may 
be obtained from the LL subband by approximating the other 

20 subbands, LH, HL, and LL, with zeros and applying the 
inverse DWT to obtain V, This technique may then be 
appUed to the Y plane to perform upscaling, by letting the LL 
subband of the upscaled image comprise the Y plane and 
appending three zero subbands, LH, HL, and HH, as illus- 
trated in FIG. 4. It is noted that a constant factor, s, is applied 
depending upon the DWT filters applied, where s is the 
absolute sum of the low pass filter coefficients for the inverse 
DWT. This parameter may, in some embodiments, be 
dynamically modifiable depending on the applied filter and 
application criteria. Thus, the Y plane may be upsampled to 

30 generate a 2Mx2N sized Y-plane by applying an inverse 
DWT to the original Y plane with artificially generated LH, 
HI^ and HH subbands. This technique for performing this 
upscaling is described in greater detail in a prior patent 
application "A DWT-based Up-Sampling Algorithm Suit- 

35 able For Image Display In An LCD panel" by Acharya, filed 
on Aug. 5, 1998, U.S. patent application Ser No. 09/129, 
728, assigned to the assignee of the present invention, 
although the invention is not limited in scope in this respect. 
As previously indicated any one of a number of techniques 
to perform the upscaling may be employed. Furthermore, 

^ other approaches to apply the inverse DWT to upscale may 
also be employed. 

It is noted that the computational complexity of this 
approach and the amount of storage space utilized is sig- 
nificantly less than the previous approach. As disciLssed 

45 above, the total number of multiplications for color conver- 
sion is 9MxN and the total number of operations for 
upscaling in this embodiment is KMxN. Hence, the total 
number of operations for this particular embodiment is 
(k+9)MxN, which is less than one-third of the approach 

50 previously described in terms of computational complexity. 
Likewise, the storage space employed to implement this 
approach is 6MxN or about half of the approadi previously 
described. 

As previously indicated, an embodiment of a method of 
55 upscaling color image in accordance with the invention 
provides a number of advantages. A computationally com- 
plex color upscaling scheme applied to the full image is 
avoided totally and, instead, an upscaling scheme is applied 
to a reduced set of data. Likewise, dropping rows and 
columns from the upscaled full color image is also omitted. 
The reduction in computational complexity is more than 
66%, and in memory storage space is about 50%. Therefore, 
this approach may be applied to virtually any digital signal 
processing architecture or similar computing engine and 
may reduce computational complexities suitable for real- 
65 lime apphcation and low power applications, such as may be 
desirable for handheld devices, with generally no significant 
loss of signal information. 
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It will, of course, be understood that, although a particular 
embodiment has just been described, the invention is not 
limited in scope to a particular embodiment or implemen- 
tation. For example, one embodiment may be in hardware, 
whereas another embodiment may be in software. Likewise, 5 
an embodiment may be in firmware, or any combination of 
hardware, software, or firmware, for example. Likewise, 
although the invention is not limited in scope in this respect, 
one embodiment may comprise an article, such as a storage 
medium. Such a storage medium, such as, for example, a 
CD-ROM, or a disk, may have stored thereon instructions, 
which when executed by a system, such as a computer 
system or platform, or an imaging system, may result in a 
method of upscaling a color image in accordance with the 
invention, such as, for example, the embodiment previously 
described. ^5 

While certain features of the invention have been illus- 
trated and described herein, many modifications, 
substitutions, changes and equivalents will now occur to 
those skilled in the art. It is, therefore, to be understood that 
the appended claims are intended to cover all such modifi- 20 
cations and changes as fall within the true spirit of the 
invention. 

What is claimed is: 

1. A method of upscaling a color image in a first color 
space format to a second color space format comprising: 25 

transforming the color image in the first color space 
format to the second color space format; and 

applying an upscaling scheme to a subset of the data 
representing the transformed color image, the subset of 
the data being less than the whole of the data repre- 30 
senting the transformed color image and including at 
least one color plane of the transformed color image, 
the at least one color plane corresponding to one of the 
color space dimensions of the second color space 
format. 35 

2. The method of claim 1, wherein the first color space 
format is the RGB color space format. 

3. The method of claim 2, wherein the second color space 
format is the Y-Cr-Cb 4:2:0 color ^ace format. 

4. The method of claim 1, wherein the first and second 
color space formats comprise two different luminance- ^ 
chrominance color space formats. 

5. The method of claim 1, wherein transforming com- 
prises applying a relationship to at least one pixel value of 
the image in the first color space format to obtain a pixel 
value in the second color space format. 45 

6. The method of claim 5, wherein the relationship is 
applied substantially in accordance with the following equa- 
tions: 

y(/,J)=0.299/I(/^+0.5S7G(/^+0.1145(/^ 5q 

C/</,/>»-0.ie9/e(/,J)-O.331G(/^+0.500J(/^ 

Cfc(/^-0.50Q/?(/^0.419G(V)-0.0815C/,y) 

where R,G, and B denote different color planes in the RGB 
color space format, Y, Cr, and Cb denote different color 
planes in the Y-Cr-Cb 4:2:0 color space format, and I,J 
denote pixel locations. 

7. The method of claim 1, wherein applying an upscaling 
scheme comprises applying an Inverse discrete wavelet 
transform to an image decomposed into subbands, at least 
one of the subbands comprising the at least one color plane 
of the transformed color image. 

8. The method of claim 7, wherein the at least one of the 
subbands comprises an LL subband of the decomposed 
image. 65 

9. The method of claim 8, wherein the remaining sub- 
bands of the decomposed image comprise only zeros. 
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10. An article comprising: a storage medium having 
stored thereon instructions to upscale a color image in a first 
color space format to a second color space format, the 
instructions, when executed by a system, resulting in: trans- 
forming the color image in the first color space format to the 
second color space format; and 

applying an upscaling scheme to a subset of the data 
representing the transformed color image, the subset of 
the data being less than the whole of the data repre- 
senting the transformed color image and including at 
least one color plane of the transformed color image, 
the at least one color plane corresponding to one-of the 
color space dimensions of the second color space 
format. 

11. The article of claim 10, wherein the first color space 
format is the RGB color space format. 

12. The article of claim U, wherein the second color 
space format is the Y-Cr-Cb 4:2:0 color space format. 

13. The article of claim 10, wherein the first and second 
color space formats comprise two different luminance- 
chrominance color space formats. 

14. The article of claim 10, wherein the instructions, when 
executed, further result in transforming the color image 
comprising applying a relationship to at least one pixel value 
of the image in the first color space format to obtain a pixel 
value in the second color space format. 

15. The article of claim 14, wherein the instructions, when 
executed, further result in the relationship being appUed 
substantially in accordance with the following equations: 

niJ) 0.299Ji(/^587G(7,J)+0.1145C/^ 
Cfc(/^=0,50CW(/,J)-0.419G(/^0.0815C/^ 

where R,G, and B denote different color planes in the RGB 
color space format, Y, Cr, and Cb denote different color 
planes in the Y-Cr-Cb 4:2:0 color space formal, and I,J 
denote pixel locations. 

16. The article of claim 10, wherein the instructions, when 
executed, further result in upscaling at least one color plane 
comprising applying an Inverse discrete wavelet transform 
to an image decomposed into subbands, at least one of the 
subbands comprising the at least one color plane of the 
transformed image. 

17. The article of claim 16, wherein the instructions, when 
executed, further result in the at least one of the subbands 
comprising an LL subband of the decomposed image. 

18. The article of claim 17, wherein the instructions, when 
executed, further result io the remaining subbands of the 
decomposed image comprise only zeros. 

19. A method of upscaling a color image transformed into 
a selected color space format comprising: 

appending zero subbands to a subset of the data repre- 
senting the transformed color image, the subset of the 
data being less than the whole of the data representing 
the transformed color image and including at least one 
of the color planes of the transformed color image so 
that the at least one of the color planes forms an LL 
subband of a decomposed image and the appended 
subbands form LH, HL, and LL subbands of a decom- 
posed image; and 

applying the inverse discrete wavelet transform to the 
decomposed image so as to form the upscaled cobr 
image in the selected color space format. 

20. The methodof claim 19, wherein the selected color 
space format coinprises the Y-Cr-Cb 4:2:0 color space 
format. 

mo*** 
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